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Abstract: This paper aims to provide some statistics concerning the science performance of a 
group of 59 8 lh graders, studying in three different classes (may be regarded as different study 
groups). Eight science items were used, two from each content domain: chemistry, biology, 
mathematics and physics. These items were taken from the 2007 edition of the TIMSS test and 
slightly adapted. Students were graded separately for a correct answer and for the explanation why 
they chose/gave the answer. Results of the applied test are presented in a graphical format. The 
authors did not develop a grading system and did not use the one provided by the TIMSS 
documentation because the number of items used is mall, and a parallel between the results in this 
study and the TIMSS would not be valid. A very good percentage of the students chose/gave the 
correct answer but a much lower percentage provided the correct and complete explanation for this 
answer. Also, from the interpretation of the results it seems that boys score slightly higher than 
girls and that the mathematics content was more accessible. 
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1. Introduction 

In the context of globalization and opposition to globalization, education systems are not following, de 
facto, to achieve the degree of homogeneity achieved by economical systems. In the case of Romania 
this seems to be especially obvious when earlier cycles of education are considered, as the curricular 
contents have not yet been brought up to date with some of the leading countries of the European 
Union. One of the ways to evaluate and compare students in different countries and thus evolving in 
different education systems is through international tests, such as TIMSS and PISA. These types of 
tests aim to asses the degree to which the students have assimilated the science content they were 
expected to learn and the stage of the cognitive development they are in. In providing complex 
answers the student must go from just possessing information (knowing), to applying it and 
reasoning (Gonzales et ah, 2008). 

Based on results from tests used periodically to evaluate perfonnances of students in 8 th grade in the 
world, Romania is not placed in one of the lead positions. After the evaluation of the Science TIMSS 
2007, Romania was ranked 28 th with a score of 462 out of 1000, while the average score was 500 and 
the maximum score, belonging to Singapore, was 567. Also, a decrease by 8% in the TIMSS score 
was registered, as the score in the 1995 TIMSS was 471 (Gonzales et ah, 2008). 

Results of this particular test cannot be compared to results from previous TIMSS or PISA tests due to 
statistical inconsistency: this test consisted of a smaller number of items and was applied to a smaller 
numbers of students. However, some conclusions can be drawn from analysis of the results of such a 
test. 

This paper is organized as follows: section 2 will describe the method used to select and interpret the 
items, in section 3 the results are presented and discussed and conclusions are drawn in section 4. 

2. Method 

A test consisting of several items was given to three classrooms of eight graders. For the classes the 
following notations were used: class 1 (21 students), class2 (18 students) and class3 (20 students). 
From a total of 59 students, 28 are girls and 31 are boys. The test has eight items, representative for 
the content domain of chemistry, biology, mathematics and physics (two items for each). Each item 
and the key are presented in the Appendix at the end of this paper. 
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For the remainder of this paper, the answers are coded as follows: 

Chemistry 1: Answer - 1, Explanation - 2. 

Chemistry2: Answer - 3, Explanation - 4. 

Biology 1: Answer - 5. 

Biology2: Answer - 6, Explanation - 7. 

Mathematics 1: Answer - 8. 

Mathematics2: A1 - 9, A2 - 10, B - 11, C - 12. 

Physics 1: Answer - 13. 

Physics2: 1 - 14, 2 - 15, 3 - 16, 4 - 17, 5 - 18, 6 - 19. 

Each item of the test is intended to evaluate the development of one or more specific cognitive 
domains. Table 1 should be correlated with the graphs presented in the next section for a correct 
assessment of the situation. 

Knowing includes the totality of tools needed by the student for him to function scientifically. Of 
interest in this study are: recognizing and using scientific vocabulary, symbols, abbreviations, units, 
describing science processes to demonstrate knowledge of properties. Also, items are focused on 
demonstrating knowledge of the use of scientific apparatus, tools, equipment, procedures, 
measurement devices and scales. Applying refers to the student’s ability to apply knowledge and 
conceptual understanding to solve problems or answer questions (Mullis et ah, 2007). In the case of 
the items proposed, the students should be able to interpret relevant textual, tabular or graphical 
information in order to provide the correct answer. Reasoning goes beyond the usual cognitive skills 
required by solving routine problems. In the particular case of the items proposed, reasoning would 
lead to establishing a connection between conceptual knowledge already possessed by the student and 
to constructing arguments to support the correctness of offered solutions (Gonzales et ah, 2008). 


Table 1. Items and the development of what cognitive domain they are observing 


Item 

Cognitive domain 

Chemistry 1 

Applying 

Chemistry2 

Reasoning 

Biologyl 

Knowing 

Biology2 

Reasoning 

Mathematics 1 

Reasoning 

Mathematic s2 

Reasoning 

Physics 1 

Applying 

Physics2 

Knowing, Applying, Reasoning 


3. Results and interpretation 

Results of the test are given in the two graphs presented below, (Figures 1 and 2), and organized as a 
statistics for the percentage of students that provided the correct answer. Note that only correct 
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answers (for questions with short answers required) and complete answers (for questions requiring 
explanations) were numbered. Incomplete answers were not taken into account by this statistics. 



□ total(59) 

□ class'! (21) 

□ class2(18) 

□ class3(20) 


question 


Figure 1. Percentage of correct answers plotted as a function of the study group. 

Figure 1 also takes into account different classes, meaning different teachers. It cannot be said that the 
differences are big enough to draw any conclusion regarding differences in the quality of their 
teaching. 



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 


□ girls(28) 

□ boys(31) 


question 


Figure 2. Percentage of correct answers as a function of gender 

According to results presented in (Mullis et ah, 2007), Romania has an 8% difference in favor of 
males. The numbers of students that took the test is not large enough to be used to produce a valid 
statistics while this issue is concerned, however the graph in Figure 2 might lead to some insight. 
Overall, the boys did score better than the girls. Answers 3, 6, 8, 9, 10, 13, 15, 18 present a clear bias 
towards the boys. However, since all the cognitive domains are tested by these questions, a definitive 
conclusion cannot be fonnulated. 

Another typical trend in international tests is to compare the performance in sciences (chemistry, 
biology and physics taken as a whole) to the perfonnance in mathematics. Such an analysis is also 
included in this study, Figure 3. 
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□ science 
■ math 


Figure 3. Percentage of correct answers in science vs. mathematics 

In Figure 4 a general comparison between the four contents represented in the test is presented. From 
figures 3 and 4 it is clear that the students found the mathematics items more accessible. This result 
could be based on a number of factors, including: 1) it is a common practice in Romanian schools, 
especially in 8 th grade, that the mathematics content be emphasized. This results in the fact that 
students work harder at mathematics. 2) Items representative for the science group were more or less 
based on practical reasoning and experimental skills, which seem to be underdeveloped due to various 
causes. 3) Again, the low number of students and items may lead to poor statistics. However, the 
difference in correct answers in mathematics vs. sciences is almost 20% percent, well above what 
could be considered as statistical indeterminacy. 



□ biology 

■ chemistry 

□ physics 

□ maths 


Figure 4. Percentage of correct answers from the total of 59 students) plotted for the different content areas in 

the test. 


4 . Conclusions 

Some very interesting conclusions can be drawn from the data obtained, even though from a statistical 
point of view, the test needed to be applied to a larger number of students, from different schools. 
Also, the subsequent interpretation of the accessibility of one content area in favor of another would 
have been more reliable had there been a larger number of representative items. 
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Although students were informed that the grades will not affect their scholar situation and that they 
also had the option not to sign the papers, it is obvious that some of them copied part of the answers 
from colleagues. 

Some of the students who obtained high scores on the first few questions exhibited what could be 
called “tiredness” when tackling with the last item. Answers to this last question ranged from complete 
and correct, to incomplete, to funny (e.g. “I believe that the king should relax a little, it’s only a 
crown”), to completely wrong or without even an attempt at solving. 

Only 2 students out of the 59 total provided the correct units for their result at number 14, although 
almost tree quarters calculated the correct numerical value. They applied the fonnula given to them in 
the text, but did not consider the actual scientific context. 

A very good percentage of the students chose/gave the correct answer but a much lower percentage 
provided the correct and complete explanation for this answer (Figure 5). The causes for this situation 
may be identified from the following: one, the students copied the answer from one another and two, 
the latter stage of their cognitive skills, reasoning, is not yet fully developed. 

There was a 20% difference in correct answers in mathematics vs. science in favor of mathematics. In 
order to explain, confirm/deny this conclusion, better statistics may be required. 

The problems identified after interpretation of this test may be addressed in time. A few steps in the 
good direction include changing the importance given by teachers to the amount of infonnation 
students know by heart in favor of the quality of their reasoning. As Feynman said “ You can know the 
name of a bird in all the languages of the world, but when you're finished, you'll know absolutely 
nothing whatever about the bird... So let's look at the bird and see what it's doing — that's what counts. 
I learned very early the difference between knowing the name of something and knowing something”. 

Another such step may consist in a higher degree of interdisciplinary connections, especially for the 
age group discussed in this paper, without cancelling the specific of each science topic. Students have 
to see mathematics is useful for elegantly describing the wonders of physics, chemistry and biology 
are not just about sulfur and bugs, but it’s a way for conservation laws in physics to come to life and 
color. 

There are many more steps that can be taken and good teachers have been taking them. But the need is 
that such educational strategies are not assumed a priori, by regulated by laws and implemented 
accordingly. 



Figure 5. Percentage of correct short answers vs. percentage of correct and complete explanations. 

As a side note, there were many worrying mistakes in basic grammar and vocabulary. Such mistakes 
were found even in tests that individually scored more than 90%. 
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Appendix 

The test items with key are presented here. 

ChemistrvK ril, B12): This item requires that the students choose the correct answer from a series of 
three possible answers and explain their choice. Two distinct objects A and B are shown on a balance 
weighing a total of 100 g. Another figure shows the same object, placed together in one recipient and 
the student is asked what the scale of the balance shows. 

Key: Answer: 100 g. Explanation: Because the mass of the reactants equals the mass of the products. 


Chemistry2([2], pagel73, number 4): This item requires that the students choose the correct answer 
from a series of three possible answers and explain their choice. Three jars of different volumes are 
shown and in each of the jars there is a lit candle. Jar X has volume Vx, jar Y has volume Vy and jar Z 
has volume Vz, with Vx=Vy. Jars Y and Z have a lid on. The student is asked to say in which one of 
the jars the candle will go out first and to explain. 

Key: Answer: jar Z. Explanation: Because fire needs oxygen to stay lit. When the lid is on, Z’s flame 
will go out first because it has less air than Y. 


Biology! (111. B8): This item requires that the students name the three remaining evolutionary stages 
of a moth. The stage of “adult moth” is provided to them. 

Key: egg - caterpillar - pupa - (adult moth) 


Biologv2_([2], page 171, number 2): This item requires that the students choose the correct answer 
from a series of five possible answers and explain their choice. The students are told that a little girl 
wants to make an experiment about what flowers need to grow. A first picture is shown of an 
experimental setup. The students are then asked which one of some other five images of different 
experimental setups the girl needs in order to prove her hypothesis and to explain their choice. 

Key: Answer: the fifth. Explanation: Because in order to prove/disprove her theory she must use 
identical setting except without the element from her theory. 


MathematicsK ril, B3): This item requires that the students provide a correct numerical answer. A pie 
diagram and a column diagram concerning the number of boys and girls from two classrooms are 
shown. The student is asked to say how many more girls are in one class compared to the other. 

Key: 14 


Mathematics2 ( f21. page 153, number 3): This item requires that the students provide three correct 
numerical answers to a problem of reduced complexity and to provide a general algorithm to solve 
similar problems with increased complexity. Namely, starting from a sequence of figures such as the 
one below 


Figure 1 

to say how many small triangles will be in A) the 3 ld and 4 th figure, B) 7 th figure and C) give a general 
algorithm to compute how many small triangles will be in the 50 th figure. 
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Figure 2 Figure 3 
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Key: A. (Al: 18, A2: 32), B: 98, C: Multiply the figure by itself and then multiply the answer you get 
by two. 


Physics! HI 1. B13): This item requires that the student chooses a correct answer. The student is 
provided with the definition of work and then he is asked in which of the following illustrated 
situations the man is doing work. The illustrations show: A) a man holding a heavy object, B) a man 
pushing against a wall, C) a man pushing a cart up a ramp and D) a man reading a book. 

Key: C 


Physics2([2], page 176, Metal crown): This item is more complex. It is based on the story of a king 
that gives a block of pure metal to his jeweller and asks that he makes a crown out of it. After the 
crown is made, the king is unsure of the jeweller’s honesty and asks his scientists to determine if the 
jeweller used only the pure metal to make the crown. The students are asked a series of questions that 
require either short answers either explanations. 1. Being given the block’s dimensions and mass they 
are asked to calculate the block’s density. 2. Next, the volume of the crown needs to be determined. 
Given some experimental setup, the students are asked to describe a way they would obtain the 
volume in the laboratory. 3. Using the procedure described at 2, the scientists measured the volume 
five times, obtaining different values shown for the students in a table. The students are asked to give 
an explanation of why the scientists measured the volume five times. 4. The students are told that the 
scientists reported to the king that the crown had a certain density. They are then asked to show how 
the scientists used the values in the table to obtain the value they reported to the king. 5. A table of the 
densities for different metals is given. The students are asked to say what metal is the block of metal 
most probably made of. 6. The students are asked what they would report to the king concerning the 
crown: what metal or mixture of metals is it made of? 

Key: 1. 19.2g/cm 3 , 2. Fill the beaker with enough water to cover the crown. Add the crown and mark 
the side of the beaker where the water level is. Then take the crown out. Use the graduated cylinder to 
add little bits of water until the level comes back up to the mark. That is the volume of the crown. 3. 
Because there is experimental error. So, measuring it 5 times you can calculate the average to know 
how much error there is. 4. They added together all of the densities and then divided by 5 to get the 
average. 5. Gold, because it has the closest density. 6. The jeweller used some silver as well as some 
gold. 
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